Development and validation of a point‐of‐care nursing mobile tool to guide the diagnosis of malnutrition in hospitalized adult patients: a multicenter, prospective cohort study

Abstract Malnutrition is a prevalent and severe issue in hospitalized patients with chronic diseases. However, malnutrition screening is often overlooked or inaccurate due to lack of awareness and experience among health care providers. This study aimed to develop and validate a novel digital smartphone‐based self‐administered tool that uses facial features, especially the ocular area, as indicators of malnutrition in inpatient patients with chronic diseases. Facial photographs and malnutrition screening scales were collected from 619 patients in four different hospitals. A machine learning model based on back propagation neural network was trained, validated, and tested using these data. The model showed a significant correlation (p < 0.05) and a high accuracy (area under the curve 0.834–0.927) in different patient groups. The point‐of‐care mobile tool can be used to screen malnutrition with good accuracy and accessibility, showing its potential for screening malnutrition in patients with chronic diseases.


INTRODUCTION
Malnutrition is a major global issue, with WHO statistics indicating that 828 million people are affected.It has a huge socio-economic impact on countries, and the diagnosis and treatment of malnutrition is a major challenge.Research has shown that among hospitalized elderly patients in China, the proportion of malnutrition is between 32.3 and 49.8%. 1 Malnutrition has been linked to a number of negative effects on patients with chronic diseases such as cancer and diabetes. 2,3In fact, it is estimated that about 10% to 20% of cancer patients die from malnutrition, not cancer itself. 4Additionally, malnourished diabetes patients have a higher occurrence of infection or foot injuries than their well-nourished peers. 5This is a stark reminder of the importance of proper nutrition in order to maintain good health and prevent the onset of serious illnesses.
Early diagnosis can prevent malnutrition and its consequences. 6To assess the nutritional status of patients, nutritional scales are typically used.The Patient Generated Subjective Global Assessment (PG-SGA) is a widely used nutritional assessment and screening tool for cancer patients, incorporating both subjective and objective parameters. 7PG-SGA short form (PG-SGA SF) was a shortened form of PG-SGA.Compared as PG-SGA, PG-SGA SF are thought easier to be understand and can be completed in short time, but their diagnostic efficiency is close. 8The Pearson correlation is calculated as the product of the covariances divided by their standard deviations. 9he Nutrition Risk Screening 2002 (NRS2002) is another nutritional risk screening method recommended by the European Society of Parenteral Nutrition (ESPEN) for hospitalized patients. 10,11Some objective examination methods were used, such as malnutrition imaging, function, and biomarkers, but these methods are often not in the field due to the economic or practical limitations. 12Unfortunately, malnutrition screening is not widely practiced due to lack of awareness or experience in dealing with it.A survey of 20,000 people in 25 European countries found that only 52% of hospitals had a routine program of malnutrition screening on admission. 13Thus, hospitals around the world need a set of portable, convenient, accurate and automatic diagnostic tools for screening malnutrition.
The utilization of facial data as a proxy for assessing an individual's body mass index (BMI) has garnered significant attention.In 2010, researchers used 2-dimensional facial imagery to evaluate BMI, utilizing metrics such as width-to-height ratio, perimeter-to-area ratio, and cheekto-jaw-width ratio. 14,157][18] However, it should be noted BMI is a poor indicator of health outside very specific demographics, thus more specific indicator should be established for screening the malnutrition.
The development of a digital smartphone-based selfadministered tool for nutritional risk screening for hospitalized patients has been explored in few studies. 19To address this issue, we developed R+ Dietitian, a digital smartphone-based self-administered tool.R+ Dietitian enabled us to collect data from four different hospitals in China, including nutritional scales and facial screens. 20he aim of this study was to develop and validate a point-of-care mobile tool to automatically extract and analyze facial features to screen malnourished patients with chronic disease.We also examined the suitability of NRS2002 and PG-SGA SF questionnaire, and if any local features, such as eye contour, are more suitable for prediction (Figure 1).

Characteristics of included participants
We used the R+ Dietitian software to prospectively collect facial data and questionnaire responses from 619 patients.The data collection period was from April

Classification criteria and feasibility analysis
We conducted a Pearson correlation analysis to determine the correlation between the scores obtained from the two systems.The analysis showed a significant correlation (p < 0.05; Table S1) in all groups.In the real world, the NRS2002 threshold score of three indicates potential malnutrition risk, while the PG-SGA SF threshold scores of 2, 4, and 9 indicate normal nutrition status, potential risk, need for nutritional intervention, and severe malnutrition, respectively.Our subsequent analysis of the distribution of scores (Figures S1 and S2) revealed that the mean value of PG-SGA SF corresponding to the NRS2002 threshold score of 3.0 was 5.079, similar to the PG-SGA SF threshold score of 4.0, indicating the need for nutritional intervention.Furthermore, the mean value of PG-SGA SF corresponding to the NRS2002 threshold score of 0.0 was 2.562, indicating good nutrition, which is similar to the PG-SGA SF threshold score of 2.0.Based on these findings, we eventually chose the PG-SGA SF scores of 2.0 and 5.0 as the three classification thresholds, representing normal nutritional status, malnutrition, and severe malnutrition. 21

Comparison of multiple models with different features
After five-class classification models and 50 rounds of training (Table S2), the results showed that the model using ocular features achieved higher area under the curve (AUC) values than the model using full-face features excluding eyes in the external verification set (with the exception of KNN in the external verification cohort of Other Inpatient Group; Table 2 and Figure S3), higher accuracy (with the exceptions of XGBoost in the training cohort of Cancer Inpatient Group under the training cohort and Extratree in the external verification cohort of Other Inpatient Group; Table 2 and Figure S4), as well as higher sensitivity (with exceptions of Adaboost in the Other Inpatient Group; Table 2 and Figure S4).

Development of face-eye weighted model
Based on the face-eye weighted features, we constructed a back propagation (BP) neural network model, which we evaluated on the Cancer Inpatient Group, Other Inpatient Group, and All Inpatient Group using fivefold crossvalidation and an external verification set.The average AUC (Figure 2) obtained from the fivefold cross-validation set was 0.886 (CI 0.843-0.930),0.834 (CI 0.764-0.904),and 0.927 (CI 0.899-0.955)for the Cancer Inpatient Group, Other Inpatient Group, and All Inpatient Group, respectively, with the corresponding AUC obtained from the external verification set being 0.860 (CI 0.817-0.904),0.843 (CI 0.796-0.889),and 0.887 (CI 0.829-0.944).The negative predictive value (NPV) and positive predictive value (PPV) for all three groups were not less than 0.75, the sensitivity was not less than 0.8, and the Specificity was not less than 0.6, except for the Other Inpatient Group, where the Accuracy was 0.778 (CI 0.642-0.913),but still not less than 0.8 in the other two groups (Figure 3 and Table S3).

Statistical analysis
To further validate our model's superiority, we conducted DeLong test, 22 Net Reclassification Improvement (NRI) test 23 and Integrated Discrimination Improvement (IDI) test. 24By comparing the results of the same model with different features, we found that the training results of the model using ocular features were better than those using full-face features excluding eyes (p < 0.05).For face-eye weighted model, we found that it was better than the nonweighted model in identifying malnutrition classification in these tests (p < 0.05).

Platforms establishment
We

DISCUSSION
This study aimed to develop and validate a malnutrition screening tool called the MSE in several Chinese hospitals to evaluate its suitability for assessing malnutrition in inpatients.Traditionally, nutritional assessment relied on user-based properties such as NRS 2002 and PG-SGA SF questionnaires.In contrast, MSE utilizes automated facial expression analysis to automatically detect malnutrition using eye attention enhancement, which shortens the malnutrition assessment to three seconds in real-world settings.Additionally, MSE can be operated from a mobile device, offering potential benefits for long-term nutritional information management of patients.The combination of these properties offers a new option for doctors and patients to improve their nutritional assessment workflow, potentially leading to changes in clinical practice to address current challenges around malnutrition screening in areas without specialized dietitians.Furthermore, to validate the superiority of MSE, the study conducted the net reclassification and IDI test, which showed it was better than the nonweighted model in identifying pain classification (p < 0.05; Figure S4).Finally, a system platform called MSE was developed, adaptable to both computers and mobile phones, based on the model developed (Figures S5 and S6).
It is worth noting that during physical assessments conducted using the PG-SGA SF, healthcare professionals typically only evaluate the orbital fat pad and temporal muscle in the facial region. 7,8The morphology of the periorbital region changes significantly in the case of tissue loss and is largely determined by the volume of the periorbital fat pads. 25These comprise the temporal fat pad, the preaponeurotic fat pad, the nasal fat pads, and the central fat pad.In cases of lipoatrophy, the loss of volume in these fat pads has been observed to accentuate the superior, medial, and lateral edges of the orbital rim, contributing to a sunken, skeletonized appearance. 26Due to the limitations of camera-captured images in providing depth information, we made the decision to use the orbital region and the rest of the face excluding the orbital region as two distinct features for inclusion in our study.Our analysis indicated that using the periorbital region as a feature yielded superior results with the same model.In other studies, such as the identification of fatigue in nat-ural environments, the performance of the eye model has also been shown to be superior to that of the face model, showing its potential unsegment. 27o ensure the reliability and robustness of our model, we conducted extensive analysis on patients from multiple centers and types.In the training set and the first external verification, we included multiple types of patients, such as cancer or diabetes, to enhance the applicability and universality of our model.Meanwhile, the second external verification set only included cancer patients, limiting the scope of our model but providing valuable guidance and inspiration for research on different types of patients.For cross-validation, we evaluated the performance of our model by dividing the data into five groups, using four groups for training and one group for validation.This method effectively reduced the risk of overfitting and improved the model's robustness. 28Bootstrap, a nonparametric method that does not require assumptions about the distribution of the data, was used to calculate the confidence intervals (CIs) of multiple evaluation metrics for the external verification set. 29This approach allowed us to comprehensively evaluate the performance of the model and improve the reliability of the evaluation results.In comparing models, we employed Delong test, IDI test, and NRI test to handle differences in receiver operating characteristic (ROC) curve shape, sample size, and other factors, providing a more objective and intuitive comparison result.The p value obtained by Delong-test could be used for significance testing to determine if the difference between two models was significant. 30ith the development of e-health, more and more point-of-care health app were developed and used to help medical stuffs and patients to get more convenient survice. 31,32In the field of monitoring hypertension, study have found 25 research, while two of them have proposed a contactless approach based on facial videos. 33,34n the fields of procedural pain and dementia, studies also revealed the potential of using unsegmented facialbased data. 35,36In the other field of diagnosing genetic diseases, facial-photograph-based e-health technology has also showed it potential. 37,38In the field of screening malnutrition, more and more mobile phone interventions were developed. 39,40][43] Malnutrition screening is a very important link in inpatients with various disease to improve their outcomes. 44ince the malnutrition due to various diseases has a certain performance on the patient's face, we developed and validated a malnutrition screening tool for multiple diseases.Our screening tool successfully makes the evaluation of nutritional status objective and visual, while simplifying the assessment and monitoring process for clinical doctors and nursing staff.In addition, the WeChat mini-program is compatible with various mobile operating systems, making it highly portable and accessible. 20,45here are some limitations that need to be addressed.First, the validation of the model was only conducted on tumor patients in the second external verification set, and other types of patients were not included.Furthermore, the accuracy of the model may be affected by the lighting and shooting angle, which requires image quality screening and restrictions.Additionally, the universality and applicability of the model may be limited by factors such as race and age, and thus should be taken into consideration during future development.Moreover, although the amount of data used in the study has been increased compared with previous works, it is still relatively small, and only machine learning was employed.Therefore, further research is required to expand the data scale, collect more video images, and apply more advanced methods such as deep learning to enhance the accuracy and stability of the model.

CONCLUSION
In summary, we developed, validated, and tested a facial photograph-based machine point-of-care platform for malnutrition screening with high applicability and universality.Our proof-of concept study provides a unique and generalizable model that could be used in community settings to screen for malnutrition in inpatients with multiple diseases.

Overview of the study
In this prospective, multicenter, cohort study, we aimed to develop point-of-care mobile tool to assist in the diagnosis of malnutrition during the intake prior to examination by a nurse or doctor.To achieve this goal, we divided the research into several individual parts.First, we evaluate the correlation between PG-SGA SF and NRS2002 to determine whether these two methods can be used as a predictor of patient nutrition, and which scale should be chosen.Next, we utilized a variety of machine learning models to explore whether the ocular area could serve as an enhanced region for facial recognition nutrition.Therefore, we proposed and validated a vision-based eyeface weighted model using BP neural network.Finally, we developed a system platform and a WeChat mini program called MSE, adaptable to both computers and mobile phones.

Ethics and inclusion statement
The inclusion criteria were: (1) were 18 years or older and had the cognitive ability to complete a questionnaire and (2) were able to give informed consent.The exclusion criteria were (1) condition was too severe to complete the questionnaire and ( 2) have obvious skin lesions on the face that prevented face recognition.All participants provided informed written consent to complete the nutritional evaluation and facial photograph used for research and development purposes.The research was registered in the Chinese Clinical Trial Registry (ChiCTR) with registration number ChiCTR2200063512.Informed consent statement was gained from all of the patients.

Procedures
R+ Dietitian was a mobile-phone based mini program designed to provide nutritional risk screening and dietary assessment.In a previous study, it was compared with experienced dietitians, and found to have high accuracy, sensitivity, and specificity. 20This mini program collects basic information from patients, and screens their nutritional risk.After the assessment, participants can choose whether to take a picture to provide their face data for analysis.In the four hospitals included in this research, we licensed the program to doctors or nurses to use it as a daily nutritional assessment tool.The study participants were divided into cancer inpatients, other inpatients, and normal controls based on their medical condition.
Those with noticeable skin damage were excluded from the study.The detailed procedure is described in the Supplementary Material.After completing the participants' daily nutritional assessment, doctors or nurses instructed the patients to take facial photos and explained the questionnaire results to them.The detailed procedure of this research was shown in the Supplementary Material.

Correlation between PG-SGA SF and NRS2002
To demonstrate the applicability of the nutrient classification method explored in this study, we analyzed the correlation between PG-SGA SF and NRS2002.[48]

Face detection and correction
For subsequent experiments, we performed face detection and correction on the image.Due to the nonuniform shooting pose and the presence of facial occlusion, we used OpenCV to detect and correct faces in the images to eliminate these errors.We used the Face_recognition algorithm to detect 68 facial key points, including the face contour, eyebrow arch, eye contour, nose, and mouth contour.After obtaining the coordinates of the key points, we calculated the center of the left and right eyes, respectively.We then took the line between the two centers to calculate the angle θ with the horizontal direction and rotated the image counterclockwise by θ, with the rotation center being the overall center of the face.For ease of analysis, we cropped and size-normalized each image.The cropping criterion was as follows: we divided the rotated image into top, middle, and bottom parts based on the facial key points, with the middle part's height being 40% of the distance from the center coordinates of the left and right eyes to the overall center coordinates of the mouth.The top and bottom heights were 30%, and the width was equal to the sum of the heights of the three parts.We fixed the image size at 256 × 256.

Feature extraction and dimensionality reduction
We utilized the Histogram of Oriented Gradients (HOG) descriptor to extract features, which constructs global features using local statistics of image gradient edge directions.First, we normalized the image color space and adjusted the image contrast using a Gamma value of 2.2 to eliminate the influence of shadow and illumination difference.We then calculated the gradient information of each pixel in the image, which further reduced the illumination interference while capturing the contour information.Next, we set the cell size to 6 × 6, the bin size to 8, and the angle range corresponding to each bin to 45 • .We calculated the gradient direction information of each cell based on the gradient information of each image pixel, and finally concatenated the feature vectors of all cells together to form the features of the entire image.To reduce the influence of the variation range of the gradient intensity, we normalized the gradient intensity of the block before performing series summation.Finally, we used principal component analysis to perform dimensionality reduction on the obtained features.

Comparison of multiple models
We used Adaboost, Extratree, random forest (RF), Xgboost, and KNN models to compare the classification performance of full-face features excluding eyes and ocular features in the PG-SGA SF system for facial assessment section.To crop the ocular features region, we performed face detection and correction, and obtained 68 facial landmarks.Then, we selected the region of the eye feature using the leftmost point of the left eyebrow as the upper-left coordinate, the rightmost point of the right eyebrow as the upper-right coordinate, and the third point of the nose as the height.For cropping the full-face features excluding eyes, we directly cropped the region of the eye feature from the image after face detection and correction.

Face-eye weighted model development
To enhance the classification performance of our model, we utilized a facial area segmentation and weighted approach to retain the information of full-face features.Specifically, we first divided the cropped facial image into 6 × 6 blocks and extracted their HOG features.Then, based on the position information of facial landmarks, we determined whether each block belonged to the eye region and assigned four times the weight if it did, otherwise multiplied by one time the weight.Finally, we trained the model using a BP neural network with stochastic gradient descent optimization.

Statistical analysis
To investigate the distribution of the included patient population, we utilized the average value of PG-SGA SF corresponding to NRS2002 as a result indicator and computed various statistical measures such as the median, upper quartile, lower quartile, maximum, minimum, and 95% CI.To assess the correlation between the included population PG-SGA SF and NRS2002, we employed Pearson test and used a correlation coefficient as an evaluation indicator.We considered p-values less than 0.05 to indicate statistical significance.To evaluate the classification performance of various models, we utilized six metrics, including accuracy, sensitivity, specificity, PPV, NPV, and AUC, and computed the 95% CI for each indicator.

C O N F L I C T O F I N T E R E S T S TAT E M E N T
The author Chengyuan He is a shareholder and member of the board of directors of Recovery Plus Inc., a company that is developing medical digital devices to help clinicians, Zhiwen Long is also employed by Recovery Plus Inc.. Recovery Plus Clinic is an out clinic attached to Recovery Plus Inc.As a private hospital, it does not have an ethics committee to conduct independent ethical review, but participates in this clinical trial as a sub-center of clinical trial.Recovery Plus Inc. provide financial support to the curent study.Chenyuan He and Zhiwen Long from this company contributed to the study design, data collection, data analysis, data interpretation, and writing of the report.The author's involvement with the company could potentially bias their interpretation of the results presented here.However, the author has taken measures to minimize this potential bias, including conducting independent analyses and involving other researchers with no affiliation to the company in the study design and data analysis.The other authors have no conflicts of interest to declare.

C O D E AVA I L A B I L I T Y
The software and source code are the property of Recovery Plus Inc.Any enquiry for access to these should be made directly to Recovery Plus Inc.

E T H I C S S TAT E M E N T
The study was registered in ChiCTR with registration number ChiCTR2200063512.This study also obtained approval for conducting research at West China Hospital (approval number: 20221153), Sichuan University Huaxi Hospital Tibet Chengban Branch (approval number:

F I G U R E 1
Overview of the proposed model.(A) The procedure of the point-of-care mobile tool for screening malnutrition.The device was developed based on various inpatients.Once collecting the basic clinical feature, malnutrition screening scale and facial photograph, the patients' nutrition status will be shown on mobile phone or computer.(B) The procedure of recruiting participants.(C) The procedure of the development and validation of the face-eye weighted BP network-based model.TA B L E 1 Basic information about included participants.
developed a system platform and a WeChat mini program called Malnutrition Screening Eye (MSE), adaptable to both computers and mobile phones, based on the model we developed (FigureS5).This mini-program provides clinical data collection functionality by allowing users to fill in their height, weight, disease type, weight change over time, appetite change over time, food intake change over time, dietary habits, and intake of various foods.It also enables facial data collection by capturing real-time facial images through the phone camera.Based on the embedded algorithms, it achieves nutritional recognition based on facial expressions.The program provides real-time evaluation results on users' energy deficit, expected protein deficit, energy intake, protein intake, NRS2002 score, PG-SGA SF score, and facial recognition results.Additionally, it offers users dietary task recommendations and a historical evaluation query function.This mini-program is compatible with both IOS and Android platforms.
Bootstrap method was used to determine the 95% CI of the external verification cohort.In order to facilitate the comparison of ROC curves obtained by different models under the same training cohort, we conducted DeLong test, IDI test, and NRI test, and considered p values less than 0.05 to indicate statistical significance.All statistical analyses were performed using python (version 3.9.1)and various machine learning libraries such as scikit-learn.A U T H O R C O N T R I B U T I O N SN.L., X. Z., C. H., H. S., and X. M. were involved in the study conception and design.N. L. and X. Z. extracted the data and completed the data analysis.C. H. managed data acquisition and contributed to study design.Z. L., T. S., L. Z., and T. D. contributed to data acquisition.W. C., Y. W., Z. L., T. S., L. Z., C. Y., T. D., and H. Z. contributed to data interpretation.X. W. contributed to revising and polishing the language of the paper.All authors contributed to data interpretation, writing, and reviewing the manuscript.All authors had access to all the data in the study and had final responsibility for the decision to submit the research for publication.All authors have read and approved the final manuscript.A C K N O W L E D G M E N T SThe funder of the study is a current employer of authors the C. H. and Z. L. who have had a role in the study design, data collection, data analysis, data interpretation, and writing of the report.This study was supported by grants from the Science and Technology Program of Sichuan Province (2022NSFSC0774), the National Natural Science Foundation of China (no.82273030), and 2021 Xinrui Tumor Supportive Treatment Research Public Welfare Project (no.311221042).

External testing dataset 2 (n = 77) p Value
a Including patients diagnosed as malignant tumor, such as lung cancer, breast cancer, esophageal cancer, and so on.bIncluding patients who are not diagnosed as malignant tumors but still need treatment as inpatient, such as type 2 diabetes, hypertension, heart failure, bone fracture, and so on.cIncluding participants who are not inpatients but medical staffs without basic diseases.On average, their height was 160.90 (SD 22.18) and weight was 59.22 (SD 11.74), indicating a general nutritional status.The mean NRS2002 score was 1.91 (SD 1.27) and mean PG-SGA SF score was 5.54 (SD 4.57).